Aiming at the problem of robust speech recognition for Vocal Effort (VE) variability, a speech recognition algorithm based on multi-model framework was presented. Firstly, changes of acoustic characteristics under different VE modes, as well as influence of these changes on speech recognition, were analyzed. Secondly, a VE detection method based on Gaussian Mixture Model (GMM) was proposed. Finally, the special acoustic models were trained to recognize whisper speech if the result of VE detection was whisper mode; otherwise articularoty features, in company with spectrum features, were introduced to recognize speech of the remaining four VE modes. The experiments conducted on isolated-word recognition show that significant improvement of recognition accuracy can be achieved when using proposed method: compared with the baseline system, the mixed corpus training method and the Maximum Likelihood Linear Regression (MLLR) adaptation method, the average character error rate of five VE modes is reduced by 26.69%,14.51% and 15.30% respectively. These results prove that articularoty feature is more robust than the traditional spectrum feature when confronting VE variability, and the multi-model framework is an efficient method for robust speech recognition related to VE variability.